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Description 

Device and method for detection of an object or a person in 
the interior of a motor vehicle 

The invention relates to a device and method for detection of 
an object or a person in the interior of a motor vehicle. 



In a moving motor vehicle fitted with airbags, persons 
10 displaced well forward in the direction of the instrument 

panel and children sitting in a child seat of the Reboard type 
on the passenger seat are at risk of being injured by the 
force of the inflating airbag in the event of an accident. In 
such situations advanced control systems for protecting 
15 vehicle occupants make every effort to switch off the airbag 
or to inflate it with less force in order to remove the risk 
of injuring a passenger. The preference is for using 
contactless, optical sensors to determine the positions of the 
passengers in the vehicle. 

20 

Usually a known device for the detection of an object or a 
person is arranged in the interior of a vehicle as a separate 
device from the actual control device for protecting vehicle 
occupants^ such that it is possible either to detect a 

25 passenger on the vehicle seat or to sample a danger zone in 
front of the folded airbag. In the latter case the vehicle 
interior or at least a part thereof is preferably scanned for 
the presence of an object or a person. If a child seat or part 
of a passenger's body is detected in this danger zone, the 

30 airbag is deployed only gradually or not at all. 



Such devices have for example at least one image recording 
unit and one analytical unit for the data sent by the image 
recording unit. In the subsequent method, the image recording 
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unit delivers images which in a first operating mode are 
subjected to identification by sampling or to classification. 
Classification takes place cyclically in the analytical unit, 
for example by means of an image processing algorithm. The 
5 classes for the data sent by the image recording unit may be 
for example: large person, small person, child in child seat, 
arrangement of the child seat, forward or backward facing, 
object on seat, empty seat, etc. 

10 However, in the first operating mode the analysis involved in 
classifying the occupancy of, say, a passenger seat is costly. 
This applies all the more if the whole area observed by the 
image recording unit(s) is analyzed (from the instrument panel 
to the seat back and from the seat cushion to the vehicle 

15 roof) , since this affects computing time, particularly when 
stereoscopic 3D image processing systems are used. Computing 
time is also greatly affected if a relatively high resolution 
is required; if a relatively costly seat occupancy 
classification is carried out; if the position of the head in 

20 the vehicle interior and its distance from the instrument 

panel are calculated; and/or in each case the whole image area 
is retrieved from the image recording unit (in the case of 
stereo this also involves two full images) . 

25 In the event of a collision, when the current passenger 

position must be classified or identified straight away so 
that deployment of the airbag can be controlled as necessary, 
means and/or methods of analysis requiring useful time units 
for position analysis are routinely insufficient to meet the 

30 relatively severe requirements, since such units are sometimes 
unavailable to the control system as reaction time for the 
protection of vehicle occupants . 

A method of alleviating this problem is already known in 
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which, when rapid movement is detected in part of a 
passenger's body, usually the head, the system switches over 
to a second operating mode which uses among other things a 
faster image processing algorithm to track that part of the 
5 passenger's body. Known tracking methods have the following 
properties, either alternatively or cumulatively: analysis of 
a smaller image area, known as a sub-window, around the 
position of a defined part of the body; limiting spatial 
deployment; in most cases discontinuing classification; 

10 calculating the position of the part of the body and its 

distance from the instrument panel, or alternatively, defining 
a danger zone and merely checking for the intrusion of an 
object into that zone without calculating distance; and/or 
lastly restricting the scanning area of the image sensor (s) or 

15 recording units thereby saving retrieval time. 

Until now the switch from the first operating or 
classification mode to the second operating or tracking mode 
has been controlled by the image processing algorithm of the 
20 analytical unit itself. However, there is a danger that the 
part of the passenger's body that needs to be tracked, 
particularly the head, has already left the restricted zone 
analyzed in the second operating mode, or that another item 
such as a loose object has just flown into the zone. 

25 

The aforementioned moment of danger is explained below using 
the example of an acceleration to the head at lOm/s^ during 
full braking, or about the same as acceleration due to 
gravity, and illustrated in Figure 1. 

30 

In this figure the reference character 13 designates a zone 
known as the COOP ("critical out of position") zone. An airbag 
12 arranged in an instrument panel 11 would have to be 
switched off if the head 17 of a passenger 16 were present in 
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this zone. This zone 13 is typically about 65cm from the 0cm 
rest position of the head 17. The distance to the instrument 
panel 11 from the 0cm rest position is typically 80cm. A 
device 1 for detecting an object or a person 16 in the 
interior 10 of a motor vehicle and arranged for example in the 
roof 14 of the interior 10 of a motor vehicle includes an 
image recording unit 2, such as a stereoscopic 3D camera 
system, which transmits the corresponding image data to an 
analytical unit 3. 

Using by way of example an update rate of 100ms for 
classification in the first operating mode, the situation in 
computing terms is then as follows. After 100ms the analysis 
is at instant 0ms, since the analysis itself requires 100ms. 
Thus after 100ms the 3D camera still sees no movement of the 
head. After 200ms the analysis is at instant 100ms. At this 
instant the 3D camera sees a movement of approx. 5cm, but at a 
typical measurement error of +/-2cm this is still not 
significant. After 300ms the analysis is at instant 200ms. The 
camera now detects a significant movement of 20cm. If a switch 
to the second operating mode, tracking, took place at that 
instant, the midpoint of a sub-window would have to be 
calculated by means of a suitable model in such a way as to 
set the midpoint at the 45cm position at that very instant. 
Even compared to the most recent analysis which is 100ms old 
the head has moved on by more than one diameter of a head. 
Moreover the head will reach the COOP zone within the next 
analysis cycle. Ideally a message should be sent right now to 
the airbag control device to deactivate the airbag in approx. 
50ms. For this the measurement results would need to be 
extrapolated from 20cm to 80cm - in any event a fourfold 
increase in the distance covered so far. After 4 00ms the 
analysis is at instant 300ms. The 3D camera now detects a 
movement of 4 5cm. But the head has actually been in the COOP 
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zone for about 50ms, Thus at this instant the present position 
of the head can only be calculated by means of a historic 
model. The extrapolation for this must be somewhat more than 
double the distance covered previously. 

5 

The values mentioned are recapitulated in the following table 
for the sake of clarity. 



Time : 


Actual head position 


Analyzed head position 


0 ms 


0 cm 




100 ms 


5 cm 


0 cm 


200 ms 


20 cm 


5 cm 


300 ms 


45 cm 


20 cm 


400 ms 


80 cm 


4 5 cm 


500 ms 


80 cm 


80 cm 



10 The above example clearly shows that present-day optical image 
recording units using devices for detecting an object or a 
person 

in the interior of a motor vehicle cannot begin detecting the 
acceleration until some 300ms after full braking. At this 
15 instant, however, it is actually already too late to switch to 
the second, faster tracking algorithm. Even operating the 
device exclusively in tracking mode is not possible on a 
permanent basis, in particular due to heat problems. 
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The object of the present invention, starting from the 
originally mentioned known device and the originally mentioned 
known method for detecting an object or a person in the 
interior of a motor vehicle, is therefore to take precautions 
5 which enable improved switching between the operating modes of 
the device, in particular by making this switching occur in 
more timely fashion. 

The part of the object relating to the method is achieved by 
10 means of the features which will emerge from Claim 1. The part 
of the object relating to the device is achieved by means of 
the features which will emerge from Claim 7. Further 
advantageous features and embodiments of the invention, which 
can be used individually or in combination with one another, 
15 will emerge from the sub-claims. 

The device is operated by means of an analytical unit, such 
that the device, for detecting an object or a person in the 
interior of a motor vehicle, comprising an image processing 

20 sensor and an analytical unit for the data transmitted from 

the image processing sensor, is operated in a first operating 
mode whilst an acceleration threshold value is not determined 
to be exceeded by the analytical unit and in a second 
operating mode whilst the acceleration threshold value is 

25 determined to be exceeded by the analytical unit. 

The analytical unit is preferably designed in such a way that 
the second operating mode is activated as soon as the value of 
the motor vehicle acceleration exceeds a threshold and the 
30 first operating mode is activated as soon as the value of the 
vehicle acceleration drops back below a threshold. 

Provided a sufficiently fast transmission speed is guaranteed, 
for instance less than 2 ms, the motor vehicle acceleration 
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values can be referenced by an external airbag control device. 

However, as an alternative the device itself to which the 
invention relates preferably includes an acceleration sensor. 
5 This can for example be an integral part of the analytical 

unit or of the image processing sensor. The transmission rates 
for the acceleration value are therefore advantageously as 
small as the error probabilities during data transmission, 
which may be over a line from the external airbag control 

10 device. Integrating the acceleration sensor into the device 
has the advantage on the one hand that it is independent of 
the airbag control device and any interference effects. And on 
the other hand no changes to the commonly used airbag control 
devices themselves are necessary. A third advantage is that 

15 there is no need for any additional data line, any additional 
utilization of existing data lines or any increase in 
computing power in order to send the data. 

Devices known as sensitive sensors or low G sensors have 
20 proved sufficiently suitable for use with the relatively low 
accelerations that occur during braking, and advantageously 
are sufficiently reliable in operation, but as cost-effective 
as the acceleration sensor types normally used for airbag 
control devices. Insofar as the last-mentioned control devices 
25 do not possess such a type of acceleration sensor, the data 

from the low G sensor can also be made freely available to the 
airbag control device. 

Inventively the second operating mode preferably provides the 
30 image recording unit with a sampling range which is more 
restricted than that of the first operating mode, and 
advantageously requires less computing time. 

Alternatively or cumulatively to the above, the second 
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operating mode also provides the analytical unit with an 
analysis cycle which is shorter than that of the first 
operating mode. Due to the shorter cycle time, preferably 25ms 
per update, around four times more measured values are 
5 available within the same measurement time, so that an 

extrapolation of the head position can advantageously be made 
with considerably greater accuracy. 

The image recording units used according to the invention are 
10 preferably cameras, and preferably stereoscopic 3D cameras. It 
is also possible, particularly in tracking mode, for an object 
or the position of a person to be determined with active 
irradiation of the object or person, preferably by using the 
methods of transit time measurement or triangulation . 

15 

A further embodiment of the method to which the invention 
relates provides that the value of the motor vehicle 
acceleration is polled by the analytical unit with preferably 
a cycle time of 2ms, or more particularly 1ms, which 
20 advantageously represents a significantly shorter cycle time 
than in pure image analysis. 

In the event of exceeding a threshold value, which according 
to the invention is preferably about 2m/s^ and integrated if 

25 necessary over an interval of time so as to eliminate 

interference variables originating from the vehicle, the first 
operating mode currently running is interrupted in favor of 
starting the second operating mode, such as rapid head 
tracking, preferably on the basis of the positional data from 

30 the last head position measurement. Since the last head 

position is at most 100ms old, as made clear by the example in 
the introduction to the description, and since the head has 
not yet experienced any external acceleration, the position 
will advantageously have changed by considerably less than 
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5cm. 

On the basis of the measured acceleration values and 
preferably with the aid of a comparison model, the next head 
5 position can then be estimated even more accurately during 
tracking. 

When the threshold value is no longer exceeded the second 
operating mode currently running is interrupted and the first 
10 operating mode is restarted. 

The advantages of the invention are in particular that the 
operating mode of the device can be switched considerably 
earlier, by some 200ms to 300ms, making it more timely. This 

15 appreciably increases the operational reliability of the 

device, since the physical parameter Acceleration is from now 
on measured directly and need no longer be determined 
indirectly from measurements of the passenger's position. Due 
particularly to the timely switching of operating modes, the 

20 position of the head for example can be measured in the second 
operating or tracking mode with significantly greater 
reliability than before. In addition, because the head is 
still virtually at rest when tracking starts, the tracking 
sub-window can be centered accurately on the head or some 

25 other part of the body. Lastly, because of the more timely 

switching of operating modes the head position can be measured 
with significantly greater accuracy during tracking. 

Additional features and further advantages of the invention 
30 will be described below in relation to examples of preferred 
exemplary embodiments and the accompanying drawings. 



These contain the following diagrams: 
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Fig. 1 



the forward displacement of a passenger's head during 
braking; 



Fig. 2 



the device to which the invention relates, having an 
acceleration sensor arranged within the analytical 



unit; and 



Fig. 3 



the device to which the invention relates, having an 
acceleration sensor arranged within the image 
recording unit; and 



Fig. 4 



a flowchart of the method to which the invention 



relates - 



Fig.l is a diagram showing the forward displacement of a 
passenger's head in the course of braking, from a rest 
position at 0cm toward an instrument panel. An arrow shows the 
forward displacement direction and a dimension line identifies 
the individual forward displacement positions. In order to 
avoid repetition, information on the meaning, sequence and 
problems of forward displacement should be obtained by 
referring to the introductory description. 

Fig. 2 shows the inventive device 1 with an image recording 
unit 2 and an analytical unit 3 for the image data transmitted 
from the image recording unit 2. The analytical unit 3 is 
connected to an external airbag-control unit 5 for the 
protection of vehicle occupants. According to the invention 
the analytical device 3 preferably includes an acceleration 
sensor 4 . 

Fig. 3 shows the inventive device 1 with an image recording 
unit 2 and an analytical unit 3 for the data transmitted from 
the image recording device 2. The analytical unit 3 is 
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connected to an external airbag-control unit 5. Unlike the 
typical embodiment according to Fig. 2, an acceleration sensor 
4 is now an integral part of the image recording device 2. 

Fig. 4 shows a flowchart of the method to which the invention 
relates. Values for the vehicle acceleration are polled by the 
analytical unit 3 with a cycle time of 1ms to 2ms. If these 
values exceed a threshold value which can be filtered as 
necessary and is preferably around 2m/s^^ the classification 
cycle currently running is interrupted and rapid head tracking 
is immediately started on the basis of the positional data 
from the last head position measurement. Since the last head 
position is at most 100ms old and the head has not yet 
experienced any external acceleration, its position will have 
changed by considerably less than 5cm. On the basis of the 
measured acceleration values and with the aid of a model, the 
next head position can then be estimated even more accurately 
during tracking. When the acceleration ends, the head will 
continue to move at its constant maximum speed or slow down. 
When a threshold speed is no longer exceeded the system can be 
switched back to the classification mode. 

In contrast to the prior art disclosed by document EP 0 694 
003 Bl or DE 40 05 598 Al for example^ according to which an 
25 acceleration signal from an airbag control device together 
with a measured movement of the passenger during a crash is 
used to define the deployment of an airbag, the present 
invention advantageously makes use of the idea of using an 
acceleration signal to switch between the different operating 
30 modes of a device for detecting an object or a person in the 
interior of a motor vehicle. For this the invention analyzes 
not just crash signals as in the documents mentioned above, 
but also acceleration signals such as those which occur during 
harsh braking, and transmits them to an airbag control device 
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so that they can be advantageously processed for the 
protection of vehicle occupants. 



